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Abstract 


To answer the question, What are the best ways to communicate uncertainties to public audiences, at-risk communities, and stakeholders 
during public health emergency events? we conducted a systematic review of published studies, grey literature, and media reports 
in English and other United Nations (UN) languages: Arabic, Chinese, French, Russian, and Spanish. Almost 11,500 titles 
and abstracts were scanned of which 46 data-based primary studies were selected, which were classified into four method¬ 
ological streams: Quantitative-comparison groups; Quantitative-descriptive survey; Qualitative; and Mixed-method and 
case-study. Study characteristics (study method, country, emergency type, emergency phase, at-risk population) and study 
findings (in narrative form) were extracted from individual studies. The findings were synthesized within methodological 
streams and evaluated for certainty and confidence. These within-method findings were next synthesized across method¬ 
ological streams to develop an overarching synthesis of findings. The findings showed that country coverage focused on 
high and middle-income countries in Asia, Europe, North America, and Oceania, and the event most covered was infectious 
disease followed by flood and earthquake. The findings also showed that uncertainty during public health emergency events 
is a multi-faceted concept with multiple components (e.g., event occurrence, personal and family safety, recovery efforts). 
There is universal agreement, with some exceptions, that communication to the public should include explicit information 
about event uncertainties, and this information must be consistent and presented in an easy to understand format. Addition¬ 
ally, uncertainty related to events requires a distinction between uncertainty information and uncertainty experience. At-risk 
populations experience event uncertainty in the context of many other uncertainties they are already experiencing in their 
lives due to poverty. Experts, policymakers, healthcare workers, and other stakeholders experience event uncertainty and 
misunderstand some uncertainty information (e.g., event probabilities) similar to the public. Media professionals provide 
event coverage under conditions of contradictory and inconsistent event information that can heighten uncertainty experience 


for all. 
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Highlights 


• Coverage of published studies, grey literature, media reports from all United Nations languages (Arabic, Chinese, 
English, French, Russian, Spanish). 

• Synthesis of findings across four methods: Quantitative-comparison groups; Quantitative-descriptive survey; 
Qualitative; and Mixed-method and case study. 

• Uncertainty is related to multiple facets, and is both uncertainty information conveyed in a message as well as 
uncertainty experienced. 

• Public often experiences uncertainty due to lack of information; for its reduction, it actively seeks information 
from all available sources. 

• Public should receive explicit, consistent, clearly understood uncertainty information speedily from authorities. 

• Uncertainty information leads uniformly to desirable results for the public but for some communities it may 
sometimes cause negative outcomes. 

• At-risk communities receive messages containing uncertainty information in lives that are already filled with 
many uncertainties due to poverty. 

• Stakeholders such as experts, policy makers, healthcare workers, and media professionals experience uncer¬ 
tainty and process uncertainty information similar to the public. 
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1.0 INTRODUCTION 

1.1 Background 

Communication during a public health emergency event 
is a complex process. As described by the World Health 
Organization (WHO), risk communication is “the process 
by which national and local government authorities provide 
information to the public in an understandable, timely, trans¬ 
parent and coordinated manner before, during and after a 
crisis; also promotes effective exchange of information 
and opinion among scientists, public health and veterinary 
experts during the alert phase to better assess, manage and 
coordinate preparedness and response activities” (WHO, 
2012, p. xii). The WHO also refers to it as the real-time ex¬ 
change of information, advice, and opinions between experts 
and/or officials and/or the publics who face a threat/ hazard 
to their survival, health, or economic or social wellbeing 
(WHO, 2015). Emergency public health risk communication 
is generally distinguished from non-emergency public health 
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risk communication exchanges by a combination of the fol¬ 
lowing characteristics: A perception of a fast emerging pub¬ 
lic health threat; a dramatically increased demand for 
information to protect health that often outstrips the ability 
of health authorities to provide it; a need to communicate 
with potentially at-risk populations before recommendations 
are certain; and a rapidly evolving situation in which infor¬ 
mation about the health threat and how to prevent its con¬ 
tinuation or spread is incomplete and changing (Reynolds, 
2002; Reynolds & Seeger, 2005). 

A public health emergency event, such as an earthquake, 
wildfire, flood, and emergent infectious disease, is usually 
characterized as having four major phases (Reynolds, 2002; 
Reynolds & Seeger, 2005): Preparation; onset; containment, 
which includes the peak of the emergency event; and recov¬ 
ery. Another characterization, also with four phases, but 
conceptualized slightly differently, includes: Prevention; 
readiness/ preparedness; response; and recovery. A fifth 
phase, evaluation, generally follows the recovery phase al¬ 
though it commonly occurs along with the earlier four phas- 
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es as well. 

Communication with the publics during public health 
emergency events is a complex process involving multiple 
stakeholders. The messages from authorities to the general 
public, specific communities, and other stakeholders, must 
be carefully designed to successfully influence health protec¬ 
tion behaviors. In particular, messages from authorities dur¬ 
ing the course of an emergency event must thoughtfully 
convey the uncertainties related to the scientific evidence 
and what is known about the impact and progression of the 
event. This becomes even more important as a key charac¬ 
teristic of such an event is the uncertainty the public experi¬ 
ences during the course of the event. Additionally, public 
health emergency events tend to be both local and regional 
and even global problems; thus, to fully know how to suc¬ 
cessfully communicate uncertainties in these situations, the 
political, cultural, and socioeconomic context in which the 
messages are received and understood must also be consid¬ 
ered. 

1.2 Objective 

The WHO commissioned a systematic review of the 
extant literature from multiple methodologies (quantitative, 
qualitative, mixed-methods) on best practices for conveying 
uncertainties during emergency health risk communication. 
Specifically, the objective of the systematic review was to 
address the following question: What are the best ways to 
communicate uncertainties to public audiences, vulnerable 
communities, and stakeholders? To answer the question, we 
looked at the broader phenomenon of interest of communica¬ 
tion and uncertainties inherent in events and emergencies 
with public health implications. To foreground the phenom¬ 
enon of interest that could potentially be measured, ob¬ 
served, or described in affected populations (publics, 
communities, stakeholders, etc.), we focused on strategies 
and tactics that were effective, or in the absence of evidence 
of an effect, appeared to work best to manage, contain, or 
bring about increase/ decrease in uncertainty. 

The focus for the systematic review were multiple-meth¬ 


ods data from field studies of populations that directly ex¬ 
perienced a relevant public health emergency event. Of 
interest were also data from studies of populations who may 
be likely to be affected by a relevant public health emer¬ 
gency event, particularly studies that focused on questions 
promoting individual preparedness for such events. Also of 
some interest were data from studies that addressed how 
organizations, predominantly government organizations or 
individuals employed by governments, respond to or work 
to develop risk communication messages. We not only 
sought studies that had comparison groups, but also in¬ 
cluded studies that examined concepts/ variables that may 
have an association with the concepts/ variables contained 
in the question and phenomena of interest, seeing these 
concepts/ variables to be potentially associated with uncer¬ 
tainty to find out what works and for whom and in what 
contexts. 

2.0 METHOD 1 

2.1 Process Design for Evidence Synthesis 

The process design for the multiple-methods evidence 
synthesis for the review is presented in Figure 1. Findings 
were extracted only from data-based primary studies. The 
design shows that the findings were grouped and processed 
within the type of study methodology stream and then 
brought together in an overarching synthesis of the findings 
across the methodology streams. Details of the process are 
presented below. 

2.2 Determining Study Methodology of 
Data-hased Primary Studies 

We started with the following categories for data-based 
primary studies: Quantitative randomized control trials; 
qualitative (ethnographic research, case studies, process 
evaluations, and mix-methods designs); mixed-method stud- 


1 The present systematic review is part of a larger WHO sponsored project. The method presented here is identical across all the studies 
stemming from this project. 
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Figure 1. Process Design of Synthesis of Evidence from Data-based Primary Studies 
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ies (combining different types of designs to explore a phe¬ 
nomenon of interest); observational and cross-sectional 
surveys; and grey literature reports. Using the above meth¬ 
odological groupings as a starting point, we initially identi¬ 
fied five methodological streams that best covered the 
method types found in the primary studies selected for the 
review: 

• Quantitative - randomized group comparison and 
non-randomized group comparison. 

• Quantitative - descriptive survey and similar designs. 

• Qualitative - open-ended questionnaire survey, inter¬ 
view, focus group, ethnography/ participant observation, 
and textual analysis. 

• Mixed-method - use of both quantitative and qualita¬ 
tive methods, where the different methods usually address 
different hypotheses and/ or research questions. 

• Case study - use of several methods, where usually all 
methods address the same research question and focus on 
one particular event/ person/ location. 

After a more in-depth perusal of the mixed-method and 
case study article/ reports, we did not find any appreciable 
methodological differences as both types utilized quantita¬ 
tive and qualitative methods with similar procedures. In 
consultation with the WHO methodologist consultant, we 
combined these two methodological streams. Thus, we 
ended up with the following four methodological streams: 

• Quantitative-Comparison Groups (QN-CG) 

• Quantitative-Descriptive Survey (QN-DS) 

• Qualitative (QL) 

• Mixed-Method and Case Study (MM, CS). 

2.3 English and Other United Nations 
Languages 

The primary search was for literature in the English 
language. Additionally, we conducted searches for studies 
published in the other United Nations (UN) languages as 
well, which included Arabic, Chinese, French, Russian, 
and Spanish. 

As seen from Figure 1, we followed the same process for 
both English and other UN languages articles/ reports for 
data extraction from individual studies and synthesis of 
findings within methodological streams. That is, the indi¬ 
vidual studies from Arabic, Chinese, French, Russian, and 
Spanish were grouped into the four methodological streams, 
irrespective of the language, after which synthesized find¬ 
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ings were generated within each methodological stream. 

We did not completely translate Arabic, Chinese, French, 
Russian, and Spanish language studies into English. Por¬ 
tions of the studies were translated into English as needed 
to meet the requirements of the review. As the other UN 
language findings from individual studies came from studies 
that were only partially translated into English, we treated 
these findings as a separate “sub-stream” at the time of syn¬ 
thesis of findings within methodological streams. 

2.4 Information Sources for Literature Search 

We conducted a search for published primary studies 
using the University Library Summon function, which in¬ 
dexes all holdings in the library, Google Scholar, and gen¬ 
eral Google search. We also searched within individual 
databases including: Communication and Mass Media Com¬ 
plete (CMMC); Cumulative Index of Nursing and Allied 
Health Literature (CINAHL); CINAHL Complete; Elsevier; 
JSTOR; Psychlnfo; PubMed/Medline-National Library of 
Medicine (NLM); Web of Science; and WHO databases. 

Native readers of Arabic, Chinese, French, Russian, and 
Spanish who were fluent in English conducted the search for 
non-English language primary studies. In addition to the 
information sources noted above, the following sources were 
also searched for each language: Arabic, Al-Manhal, and 
Dar-Al-Manduma; Chinese, CNKI (China National Knowl¬ 
edge Infrastructure), and Wanfang Patent Database; French, 
Archive ouverte UNIGE, Cairn.info, Government of Can¬ 
ada publications, HAL archives ourvertes, La Houille Blanc, 
Persee.fr, and Revues.org; Russian, Cyberleninka.ru, Mgi- 
mo.ru/library/ehd, Msu.ru/info/struct/dep/library, and 
Nbmgu.ru; and Spanish, CONACYT, Cuiden, and Public 
Health Institute Mexico. In addition, persons familiar with 
non-English language publications were suggested by the 
WHO and they were solicited for suggestions for relevant 
studies. 

The search for grey literature in all languages used 
Google Scholar and general Google search as the main in¬ 
formation sources. In addition, an experienced librarian at 
the National Hazards Center library at the University of 
Colorado-Boulder, United States conducted a search spe¬ 
cifically for grey literature. The search was conducted in 
close consultation with a team member who was physically 
present on location. 
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2.5 Literature Search Strategy, Search Terms, 
and Search Inclusion and Exclusion Criteria 

We adopted a two-phase strategy for literature searching. 
In the first phase, we did a general search that was intention¬ 
ally broad in scope. In the second phase, a search focused 
narrowly on the objective of the present review was con¬ 
ducted. 

We used the search terms shown in Table 1. Not all terms 
worked in all databases; therefore, thesauri were consulted 
for each database to find synonyms, if they existed, for each 
term, or any functionality that allowed the word to be “ex¬ 
ploded” or “expanded.” 

The following broad inclusion criteria were used in the 
search for literature to capture empirical research of all 
methodological types: 

• Research related to the practice of risk communication 
and the process of disaster management with no preference 
for any specific emergency or health hazards. 

• Research within the viewpoint or scope set by the risk 
communication field related to: trust, uncertainty, communi¬ 
ties, health, misinformation, health protection, media (in¬ 
cluding social media), messages, and stakeholders. 

The following exclusion criteria were used in the search 
for literature to keep a focus on public dissemination of 
uncertainty information: 

• Research in organizational risk communication and 
disaster management such as technology failures. 

• Research outside of the specified scope of the study, 
such as laboratory studies and those related to chronic dis¬ 
ease, lifestyle, or personal living/ attributes (such as per¬ 
sonal health, mental health, etc.). 

• Studies published before 2003. This was a WHO stipu¬ 
lation based on a need for current research only. 

2.6 Article/ Report Selection 

The hits generated by the literature search process were 
narrowed to select data-based primary articles and reports. 
The general process for selection of the articles/ reports for 
all languages was in several steps that were in two stages, 
broadly conforming to the Preferred Reporting Items for 
Systematic Reviews and Analyses (PRISMA) process (Mo- 
her, Liberati, Tetzlaff, Altman, The PRISMA Group, 2009). 
After initial hits were pulled through the searches, the arti¬ 
cles were screened for relevance to the review topic and ob¬ 
jective at each stage. In the first stage: 


• The hits obtained using a search were scanned by read¬ 
ing their title and abstract or summary to assess relevance 
to risk communication during disaster/ emergency events; 

• After scanning, the hits that were judged as related to 
risk communication during disaster/ emergency events were 
quickly read as full-texts and downloaded if found still 
broadly related; 

• The downloaded full-texts were read carefully, and if 
found related to the objective and phenomena of interest of 
the present review, were selected. These included, both aca¬ 
demic and grey literature, data-based studies, reviews, guide¬ 
lines, and media reports. 

In the second stage: 

• The full-texts of the selected articles and reports were 
again read and this time categorized as a data-based pri¬ 
mary study or not. This included the grey literature. 

• If an article/ report was a data-based primary study, it 
was further judged for different levels of relevancy to the 
review objective and phenomena of interest (see Lewin et 
al., 2015 and Noyes et al., 2018, for details of the relevancy 
criteria). Studies were judged to have direct relevance (i.e., 
directly mapped onto phenomenon of interest); indirect rel¬ 
evance (i.e., corresponded with some aspects of the phenom¬ 
enon of interest); partial relevance (i.e., a part of the issue of 
interest or population was addressed but not all); or unclear 
relevance (i.e., unclear whether underlying data were rele¬ 
vant) with the review topic. A data-based study that was 
judged as directly, indirectly, partially, or uncertainly rele¬ 
vant (as opposed to not relevant at all) was selected for ex¬ 
traction of its key findings. Only these relevant (direct, indirect, 
partial, unclear) primary study articles/ reports were used to gener¬ 
ate the systematic review for this report. These included studies 
used quantitative, qualitative, mixed-method, and case study 
methods. 

To summarize, the article/ report selection process oc¬ 
curred in two broad stages. In the first stage, all literature 
that was related to disaster/ emergency risk communication, 
and review objective and phenomena of interest was selected. 
In the second stage, this literature was narrowed to select 
only relevant data-based primary study articles/ reports us¬ 
ing quantitative, qualitative, mixed-method, and case study 
methodologies. 

The first stage of the search and selection for English 
language articles/ reports was conducted by an experienced 
librarian with subject-matter expertise in the discipline of 
communication. Two training and norming sessions were 
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conducted with the librarian. The second stage selection was 
done by all primary members of the research team, who had 
gone through a training and norming session. For ambigu¬ 
ous cases, decisions were made through discussion among 
the primary research team members. Both the first and 
second search and selection stages for other UN languages 
were done by fluent readers and writers of Arabic, Chinese, 
French, Russian, and Spanish who were also fluent in Eng¬ 
lish. Four norming and training sessions were conducted 
with this group in a group setting. In addition, individual 
training sessions were provided as needed. 

2.7 Quality Appraisal of Selected Individual 
Studies 

The individual data-based primary studies selected for 
the review were appraised for their quality using available 
tools. Quantitative control/ comparison groups studies were 
individually appraised using the Effective Practice and Or¬ 
ganisation of Care (EPOC) (2015) risk of bias tool. This tool 
provides nine criteria for assessing randomized control trials, 
non-randomized control trials, and control before-after stud¬ 
ies. Detailed information on the definitions of levels of risk 
used in this tool is available in section 12.2.2 of the Cochrane 
Handbook (Higgins & Green, 2011). 

Quantitative descriptive survey studies were individu¬ 
ally appraised using an adapted version of Davids and Ro¬ 
man’s (2014) quality appraisal criteria. This tool assessed on 
a 0 to 1 scale (0-not reported, 1-reported) the following areas: 
sampling, response rate, validity and reliability, sources of 
data, content and focus of study, and relevancy to the cor¬ 
responding question. Final ratings were determined by per¬ 
centage, as noted in the appraisal tool: weak (0-33.9%), 
moderate (34-66.9%), and strong (67-100%). We used these 
scores as indicators and not as a hard rule, and more impor¬ 
tantly the specific methodological weaknesses that were 
identified were considered in relation to how they could 
potentially impact on findings for the appraisal. 

Qualitative studies were individually appraised using 
Critical Appraisal Skills Programme (CASP) (2013) check¬ 
list. Areas of the study appraised by CASP include appro¬ 
priateness of qualitative methodology, data collection, 
relationship between research and participants, ethics, rigor 
of data analysis, clarity of findings, and value of research. 
Each area in CASP is assessed using “yes,” “no,” or “can’t 
tell.” Studies received a final rating of “high” (no significant 
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flaws), “moderate” (minor flaws impacting credibility/valid¬ 
ity), “low” (some flaws likely to impact credibility/validity), 
or “very low” (significant flaws impacting credibility/valid¬ 
ity). 

Mixed method and case study studies were appraised 
using Pluye et al.’s (2011) Methods Appraisal Tool (MMAT). 
Studies were assessed for the employed methods and meth¬ 
odological quality (i.e., qualitative, quantitative randomized 
control trials or non-randomized control trials, quantitative 
descriptive, and overall implementation of mixed methods). 
Each area in MMAT is assessed using “yes,” “no,” or “can’t 
tell.” Studies received a final rating of “high” (no significant 
flaws), “moderate” (minor flaws impacting credibility/valid¬ 
ity), “low” (some flaws likely to impact credibility/validity), 
or “very low” (significant flaws impacting credibility/valid¬ 
ity). 

Individual media reports were appraised for their quality 
using the Authority, Accuracy, Coverage, Objectivity, Date, 
and Significance (AACODS) (Tyndall, 2008) tool. Each area 
in AACODS is assessed using “yes,” “no,” or “can’t tell.” 
Studies received a final rating of “high” (no significant 
flaws), “moderate” (minor flaws impacting credibility/valid¬ 
ity), “low” (some flaws likely to impact credibility/validity), 
or “very low” (significant flaws impacting credibility/valid¬ 
ity). 

2.8 Extraction of Data from Selected Individual 
Studies 

Study characteristics and key findings along with sup¬ 
porting information were extracted from individual data- 
based primary studies of all methodological streams. We 
used the general process of reading and re-reading the full 
article, especially the abstract, results/ findings/ analysis, 
and discussion/ conclusion sections to identify the charac¬ 
teristics and findings of interest. The following study char¬ 
acteristics were extracted: method; country focus; disaster/ 
emergency type; disaster/ emergency phase; and whether 
at-risk/ vulnerable population. These characteristics were 
of interest to the WHO. The key findings and supporting 
information from each study were also extracted. The pur¬ 
pose of extraction of findings was to identify and note evi¬ 
dence of interest that mapped onto the phenomena of 
interest and the outcomes/ impacts related to the review 
objective. 

Given that only one publication (Johnson & Slovic, 2015) 
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used comparison groups (randomized or non-randomized) 
and presence of heterogeneity of studies and outcomes, a 
quantitative meta-analysis was not suitable for the review. 
As such, as recommended in Section 11.7.2 of the Cochrane 
Handbook (Higgins & Green, 2011) dealing with situations 
where meta-analyses are not possible to conduct, we followed 
a narrative summary approach (see also Popay et al., 2006) 
to extraction of findings from studies in all four method¬ 
ological streams. 

As per the narrative summary approach, each finding 
along with supporting information was extracted in the form 
of short 3-5 sentence paragraphs. The findings focused on 
the phenomena of interest broadly and the outcomes/ im¬ 
pacts specifically, and the support for each finding was in 
the form of quantitative and qualitative information. From 
quantitative studies, we extracted numerical data, such as 
means, standard deviations, and probability values. While 
extracting these data, we kept in mind whether the study 
was a group comparison (randomized, non-randomized) or 
descriptive. From qualitative studies, we extracted key phras¬ 
es, sentences, and direct quotations. From mixed-method 
and case study studies, we extracted numerical data and key 
phrases, sentences, and direct quotations as appropriate re¬ 
lated to each method. The extraction included page and 
paragraph numbers for the supporting information as well. 

2.9 Quality Assurance of Extraction of Data 
from Individual Studies 

An initial a priori codebook for extracting study charac¬ 
teristics and findings from individual studies was developed 
based on examples available from the WHO. After receiving 
feedback on a draft from team members, the WHO method¬ 
ologist, and the WHO, the document was suitably revised. 
Next, two pilot tests of the codebook were conducted, one 
for extracting study characteristics and one for extracting 
findings and supporting information, using approximately 
1% of the English language articles/ reports with three team 
members. The pilot tests generated suggestions for refine¬ 
ment and the final codebook was created after incorporating 
this feedback. After this, training sessions for the use of the 
codebook were conducted with the full research team. Using 
the codebook, the data extraction from individual studies 
was done by a team member (English language by the lead 
author of the review; other UN languages by a native reader) 
and the output was scrutinized by at least one other team 


member. The final extracted data reflected corrections based 
on the team member feedback. 

2.10 Synthesis of Findings 

2.10.1 General Process 

The synthesis of findings was done in two stages as pre¬ 
sented in the process design in Figure 1. In the first stage, 
findings from individual studies were synthesized within 
methodological streams and then these within-method syn¬ 
thesized findings were evaluated for certainty/ confidence 
using appropriate tools. In the second stage, the within- 
method synthesized findings were synthesized across meth¬ 
odological streams, taking into account the certainty/ 
confidence evaluations. 

In both the within-method and across-method stages, the 
synthesis of findings included subgroup analyses. These 
included examination of type of emergency event, phase of 
emergency event, country of emergency event, and presence 
of vulnerable population. The last two subgroups allowed 
considerations of equity in the synthesized findings. 

The synthesis of findings was done by the lead author of 
the review. The synthesis process and the synthesized find¬ 
ings were discussed with all team members in weekly meet¬ 
ings. One team member closely read the synthesized findings 
and offered critique. The synthesized findings were devel¬ 
oped and finalized based on the discussion and critique. 

2.10.2 Synthesis of Findings Within Each 
Methodological Stream 

For each methodological stream, the synthesized findings 
were created by building explanatory and higher level ana¬ 
lytical statements supported by quantitative and qualitative 
evidence from individual studies. 

For the two quantitative methodological streams, we 
again took directions from Section 11.7.2 of the Cochrane 
Handbook (Higgins & Green, 2011) dealing with results 
without meta-analyses and followed a narrative summary 
approach to synthesis of findings. For the qualitative meth¬ 
odological stream, we broadly followed the framework syn¬ 
thesis model (Barnett-Page, & Thomas, 2009; Pope, 
Ziebland, & Mays, 2000). We found this model suited to 
organize and analyze large amounts of data, which for us 
was represented by the corpus of findings and supporting 
evidence. The model is a mix of deductive-inductive 
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processes. As part of the codebook noted in Section 2.9 
above, we started with a list of a priori framework categories 
generated from review objectives and phenomena of interest 
concepts, and modified the list as appropriate based on prior 
subject matter knowledge and reading of individual studies. 
Our goal was to synthesize the findings by identifying 
themes that emerged across the findings from individual 
studies and fit the framework categories. For the mixed- 
method and case study methodological stream, the indi¬ 
vidual studies typically did not differentiate their overall 
findings based on type of methodology. For this stream, thus, 
we looked at the findings holistically and followed a broad¬ 
ly narrative summary approach. 

The assessment of certainty/ confidence of synthesized 
findings was done separately for each methodological stream 
using available tools. Quantitative-comparison groups with- 
in-method synthesized findings were assessed for certainty 
using the Grading of Recommendations Assessment, Devel¬ 
opment, and Evaluation (GRADE) approach (GRADE 
Working Group, 2004; Guyatt et al., 2011; Higgins & Green, 
2011). Findings were assessed on allocation sequence and 
concealment, baseline outcomes and characteristics, protec¬ 
tions against contamination^), presence of selective outcome 
reporting, and other possible forms of bias. Each category 
was given a rating of “low risk” (most information from 
studies at low risk of bias), “unclear risk” (most information 
from studies at low or unclear risk of bias”, and “high risk” 
(large proportion of information from studies at high risk of 
bias) More detailed information on the definitions of levels 
of risk used in this tool is available in section 12.2.2 of the 
Cochrane Handbook (Higgins & Green, 2011). Findings 
received a final rating of “high quality” (it is highly likely 
that new research will not modify the finding substantially), 
“moderate quality” (it is somewhat likely that new research 
will not modify the finding substantially), “low quality” (it 
is somewhat likely that new research will modify the finding 
substantially), or “very low quality” (it is highly likely that 
new research will modify the finding substantially). 

Quantitative-descriptive survey within-method synthe¬ 
sized findings were assessed for certainty using a tool devel¬ 
oped for the present review that was based on the principles 
of Grading of Recommendations Assessment, Development, 
and Evaluation (GRADE) as noted above. Adjustments were 
made to the GRADE process to create the tool for evaluation 
of certainty of findings from quantitative cross-sectional 
surveys that did not have comparison groups for outcomes 


of interest. There were four evaluation categories: High qual¬ 
ity (highly likely that new evidence will not substantially 
modify the study findings); moderate quality (somewhat 
likely that new evidence will not substantially modify the 
study findings); Low quality (somewhat likely that new evi¬ 
dence will substantially modify the study findings); and very 
low quality (highly likely that new evidence will substan¬ 
tially modify the study findings). The evaluation categories 
were based on factors that can reduce the quality of study 
findings: Limitations in study design or execution; inconsis¬ 
tency of results; indirectness of evidence; imprecision of re¬ 
sults; and publication bias for findings collated across 
multiple quantitative studies. 

Qualitative within-method synthesized findings were 
assessed for confidence using GRADE-Confidence in the 
Evidence from Reviews of Qualitative research (GRADE - 
CERQual; Lewin et al., 2015). Findings were assessed on 
methodological limitations, relevance, coherence, and ade¬ 
quacy of data supporting the finding. Each finding was then 
given a rating of “high confidence” (it is highly likely that 
the finding is a representation of the phenomena), “moderate 
confidence” (it is likely that the finding is a representation 
of the phenomena “low confidence” (it is possible that the 
finding is a representation of the phenomena), or “very low 
confidence” (it was not clear if the finding is a representation 
of the phenomena). 

Mixed method and case study within-method synthe¬ 
sized findings were assessed for certainty/ confidence using 
principles of GRADE and GRADE-CERQual approaches 
as appropriate. It should be noted that the adaptation of 
GRADE principles for application to descriptive quantitative 
studies and use of GRADE-CERQual principles for applica¬ 
tion to mixed-method studies has not been approved by the 
tool originators. 

2.10.3 Synthesis of Findings Across Methodological 
Streams 

We synthesized the findings across the four method¬ 
ological streams to develop an overarching synthesis of find¬ 
ings. The synthesized findings within a methodological 
stream were compared and contrasted with findings from 
the other methodological streams. Whenever the findings 
supported and amplified each other, they were combined 
into higher order findings that represented synthesis across 
the method streams. The evaluation of certainty in the with¬ 
in-method synthesized findings was kept in mind during this 
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process. 

All methodological streams did not yield the same kind 
or similar number of synthesized findings. We did not con¬ 
sider this a problematic issue as we were seeking to find the 
points of alignment of the findings across the method 
streams rather than simply merging them together, which 
would have given some methodological streams more im¬ 
portance than others. 

Within-method findings that did not contribute to an 
across-method higher order finding were analyzed themati¬ 
cally. These thematic analyses were used to uncover a nuance 
or modification to the across-method findings, which were 
then either used to create a new higher order across-method 
finding or incorporated into an existing across-method find¬ 
ing. 

A very few synthesized findings within a methodological 
stream provided evidence that countered the synthesized 
findings from other methodological streams. Whenever this 
happened, we strived to retain this finding as a separate 
finding in the final set of across-method findings or used it 
to modify an existing across-method finding. 

3.0 RESULTS 

3.1 Study Selection 

For English language literature search, almost 2900 titles 
and abstracts were scanned, of which about 1700 full-texts 
were quickly read. After this, 73 full-texts were downloaded, 
of which the 33 data-based primary studies were selected for 
data extraction. These details and those for Arabic, Chinese, 
French, Russian, and Spanish languages are provided in 
Table 2. 2 


3.2 Study Characteristics 

Table 3 provides the study characteristics (country focus, 
disaster/ emergency type, emergency phase, and vulnerable/ 
at-risk population groups) as well as method type and rele¬ 
vancy judgments for both English and other UN languages. 
There were 33 English language studies, of which six were 
directly relevant, 17 were indirectly relevant, and 10 were 
partially relevant. Of the 13 other UN languages (i.e., not 
English) studies, there were no Arabic, one Chinese, five 
French, three Russian, and four Spanish studies. Seven stud¬ 
ies were directly relevant and six were indirectly relevant. 
The relevancy was judged as only direct and indirect due to 
lack of sufficient clarity for the partial and unclear categories 
when applied to non-English languages. 

3.3 Quality Appraisal of Individual Studies 

Of the 33 English language studies, one was placed in the 
quantitative-comparison group stream, 10 in the quantita¬ 
tive-descriptive survey stream, 11 in the qualitative stream, 
and 11 in the mixed methods/ case studies stream. The 
studies were appraised for their quality as discussed in Sec¬ 
tion 2.7 above and the appraisals are presented in Table 4. 
For the other UN languages, a quality appraisal could not 
be determined for all the studies. This is noted as needed 
when evaluating the certainty/ confidence of the synthesized 
findings. 3 

3.4 Synthesis of Findings Within 
Methodological Stream and Evaluation of 
Certainty and Confidence 

Findings from individual studies, both English and other 
UN languages, were put into four method streams as dis- 


2 There was no grey literature included in the present review. All the grey literature identified relevant to the review question did not 
include any data-based primary studies; instead, the literature either was best practices that related the practices to the existing research 
or it was theoretical essays, that referenced data-based studies and other essays. Similarly, no media reports were included in the review 
for lack of data-based findings. Two media reports were identified for the review objective in the search for English-language news stories. 
Both reports were press releases and did not report any data. 

3 The tables for English language studies that present the quality rating, as well as relevancy and extracted findings, for each study can 
be obtained from the first author. 
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cussed above and these findings were synthesized within 
each method. Table 5 provides all details, which include the 
within-method synthesized findings, citations supporting 
each synthesized finding, evaluation of certainty/ confi¬ 
dence of the synthesized finding (as described in Section 
2.10.2 above), and explanation of this evaluation. As seen 
from the table, an individual study could support more than 
one synthesized finding and most synthesized findings were 
supported by multiple studies though a few had support from 
only one study. 

The quantitative comparison group stream had one syn¬ 
thesized finding. It was supported by a single study that 
covered an infectious disease event in the United States, with 
a focus on the onset and containment phases. No vulnerable 
populations were studied. The evaluation of certainty in the 
finding ranged from low to moderate. 

The quantitative descriptive survey stream had six syn¬ 
thesized findings. Two findings were supported by only a 
single study whereas the rest were supported by multiple 
studies. The countries covered included China, India, Mex¬ 
ico, New Zealand, Thailand, the United Kingdom, the 
United States, and general global. Bioterrorism, cyclone, 
earthquake, foodborne illness, hurricane, industrial accident, 
infectious disease, volcanic, and wildfire events were cov¬ 
ered. All four phases of an event were covered, with empha¬ 
sis on preparation. Vulnerable populations were covered in 
two findings. The evaluation of certainty in the findings 
included high (1 finding), moderate (2 findings), and low (3 
findings). 

The qualitative stream had six synthesized findings. One 
finding was supported by only a single study whereas the 
rest were supported by multiple studies. The countries cov¬ 
ered included Australia, Canada, China, general European 
Union countries, France, Iran, Japan, Russia, the United 
Kingdom, and the United States. Bioterrorism, earthquake, 
flood, hurricane, infectious disease, tornado, and general 
public health events were covered. All four phases were 
covered, with emphasis on preparation and onset, along with 
evaluation. Vulnerable populations were covered in three 
findings. The evaluation of confidence in the findings were 
high (1 finding) and moderate (5 findings). 

The mixed methods/ case study stream had eight syn¬ 
thesized findings. Two findings were supported by only a 
single study whereas the rest were supported by multiple 
studies. The countries covered included Canada, Chile, gen¬ 
eral European Union countries, Finland, France, Indonesia, 


Israel, New Zealand, Russia, Singapore, South Africa, and 
the United States. Air pollution, earthquake, flood, food- 
borne illness, industrial accident, infectious disease, land¬ 
slide, and general public health events were covered. All four 
phases of an event were covered, with emphasis on prepara¬ 
tion, along with evaluation. Vulnerable populations were 
covered in one of the findings. The evaluation of certainty/ 
confidence in the findings was high (1 finding) and moderate 
(7 findings). 

3.5 Synthesis of Findings Across 
Methodological Streams 

The across-method findings were the synthesis of the 
findings within the four method streams. Table 6 presents 
the details, which include the across-method synthesized 
findings, citations supporting a finding, and the evaluation 
of certainty/ confidence of the finding (as described in Sec¬ 
tion 2.10.2 above). 

There were total nine synthesized findings across the four 
method streams. Of these, one synthesized finding was based 
on all four methods, three synthesized findings were based 
on three methods, two synthesized findings were based on 
two methods, and two synthesized findings were based on 
just one method. The across-method synthesis sought to 
identify commonalities in themes across the method streams 
but at the same time it allowed for findings that were unique 
to not get subsumed under more general themes; this re¬ 
sulted in two synthesized findings that drew only from one 
method. 

The quantitative comparison group within-method syn¬ 
thesized finding appeared in one across-method synthesized 
finding, quantitative descriptive survey within-method syn¬ 
thesized findings appeared in three across-method findings, 
qualitative within-method synthesized findings appeared in 
three across-method synthesized findings, and mixed meth¬ 
od/ case study within-method synthesized findings appeared 
in four across-method findings. 

The certainty/ confidence of each final across-method 
synthesized finding was an aggregation of the method-spe¬ 
cific certainty/ confidence assessments of the within-meth¬ 
od findings that constituted it. The range of the aggregated 
certainty/ confidence was moderate-to-high for two find¬ 
ings, moderate for three findings, low-to-high for three find¬ 
ings, and low-to-moderate for one finding. Although, no 
overall single certainty/ confidence judgment was created 
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for a final synthesized finding, based on the range of assess¬ 
ments, a judgment of moderate certainty/ confidence can be 
reasonably assigned to each finding and as such to the find¬ 
ings as a whole set. 

3.5.1 Core Aspects of Across-Method Synthesized 
Findings 

The review sought to answer the question, What are the 
best ways to communicate uncertainties to public audiences, 
vulnerable communities, and stakeholders? The review took 
the question more broadly and studied it as the phenomenon 
of uncertainty in the context of public health emergency 
events. Each of the nine final synthesized findings covered 
several features of the phenomenon and the core aspects of 
each of finding are presented next, broadly organized around 
the research question. 

1. For the public, uncertainty related to a public health 
emergency event is in terms of uncertainty information 
conveyed in messages as well as uncertainty experienced. 
The uncertainty information can be directly (or, objectively) 
in the form of numerical probabilities (“60% chance”), lin¬ 
guistic likelihoods (“strong chance”), and absence (“no in¬ 
formation”) or indirectly (or, subjectively) in the form of 
incomplete, inconsistent, and conflicting/ contradictory 
information, which all can lead to a cognitive and affective 
experience of uncertainty. Additionally, there are several 
types/ sources of uncertainty, such as uncertainty regarding 
personal and others’ safety, event knowledge, influx of non¬ 
local people, and future of village/ town, which change 
across the time course of the phases of an event. Uncer¬ 
tainty is separate from values issues, which deal with judg¬ 
ing the appropriate standards of public protection (Afifi, W., 
Afifi, T., & Merrill, 2014; Afifi, W., Felix, &, Afifi, T., 2012; 
Alipour et al., 2015; Downton, Morss, Wilhelm, Gruntfest, 
& Higgins, 2005; Doyle, Johnston, McClure, & Paton, 2011; 
Duchene, & Journel, 2004 [French]; Janmaimool, & Wa- 
tanbe, 2014; Miles, & Frewer, 2003; Podkorytov (2014) [Rus¬ 
sian]). 

2. The public’s experience of uncertainty in a public 
health emergency event is due to lack of information and to 
reduce this uncertainty it actively seeks both general and 
specific information from traditional mass media, social 
media, and interpersonal network sources (family, doctors, 
nurses, community leaders) during the course of an event. 
The uncertainty experienced by the public is associated with 
other predictors as well, including lack of: disaster manage¬ 


ment coordination and cooperation at the local level; disas¬ 
ter management plan that is interactive with the public; 
integrated management of official response to event across 
all mass and social media and other domains; and speedy, 
regular, timely, reliable, and detailed information about 
event progression. Individual difference factors such as per¬ 
ceptions of risk also are associated with experience of uncer¬ 
tainty (Acar, & Muraki, 2011; Afifi, W., Afifi, T., & Merrill, 
2014; Afifi, W., Felix, &, Afifi, T., 2012; Aldunce, & Leon, 
2007; Barengo, Tuomilehto, Nissinen, & Puska, 2011 [Span¬ 
ish]; Bird, Ling, & Haynes, 2012; Burke, & Zhou, 2009; 
Dabner, 2012; Francescutti, 2007 [Spanish]; Karan, Aileen, 
& Elaine, 2007; Muniz, 2011 [Spanish]; Skinner, & Ramp- 
ersad, 2014; Spence, Lachlan, & Burke, 2007; Taylor-Robin- 
son, Elders, Milton, & Thurston, 2009; Vallejos-Romero, & 
Onate Nancucheo, 2013 [Spanish]) 

3. The public should receive explicit information about 
uncertainties associated with public health emergency events 
in the communications sent by authorities. There is univer¬ 
sal agreement among experts and researchers regarding this, 
although a few experts/ scientists indicate that disseminat¬ 
ing scientific uncertainty to a public unable to conceptualize 
uncertainty in scientific terms can have a negative impact 
on the public’s trust of science, scientists, and scientific in¬ 
stitutions, and can lead to panic and confusion regarding the 
extent and impact of a particular event (Doyle, Johnston, 
McClure, & Paton, 2011; Frewer et al., 2003; Holmes, Hen- 
rich, Hancock, & Lestou, 2009; Janmaimool, & Watanbe, 
2014; Li, Qian, Ma, & Ge, 2010 [Chinese]; Masse, Wein- 
stock, Dessy, & Moisan, 2011 [French]; Pappenberger et al., 
2013; Quinn, Thomas, & Kumar, 2008; Taylor-Clark, Koh, 
& Viswanath, 2007; UN/FAO, 2011 [French]). 

4. For the general public, uncertainty information in 
messages provided by authorities at times of public health 
emergency events is uniformly associated with desirable 
outcomes such as reduced uncertainty about health protec¬ 
tion actions; reduced reliance on misinformation, rumors, 
and sensationalized media stories; and improved response 
to future warnings. However, when openly acknowledging 
uncertainties the possibility of some undesirable outcomes 
for some vulnerable populations needs to be kept in mind, 
such as reduction in trust in authorities and vacillation about 
evacuation (Duchene, & Journel, 2004 [French]; Gryzu¬ 
nova, 2012 [Russian]; Jakubowski, & Charpak, 2004 
[French]; Janmaimool, & Watanbe, 2014; Johnson & Slovic, 
2015; Lord, 2009 [French]; McClure, Doyle, & Velluppillai, 
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2015; Miles, & Frewer, 2003; Morss & Hayden, 2010; Pozd- 
nyakov, 2011 [Russian]; Ramos, Mathevet, Thielen, &Pap- 
penberger, 2010; Sharma, & Patt, 2012; Skinner, & 
Rampersad, 2014; Vaughan, Tinker, Truman, Edelson, & 
Morse, 2012). 

5. For vulnerable communities, the development of mes¬ 
sages containing uncertainty information about public health 
emergency events must keep in mind the whole living envi¬ 
ronment of the intended audience. People’s lives can be full 
of uncertainties due to poverty and not just because of a 
particular hazard or event. It should be recognized that is¬ 
sues of economic development and environment are just as 
central to reducing uncertainty regarding an event as mes¬ 
sages from authorities (Aldunce, & Leon, 2007; van Voorst, 
2015). 

6. The public’s understanding of some uncertainty infor¬ 
mation associated with public health emergency event like¬ 
lihood estimates is error prone and this error is true of 
various stakeholders such as experts (scientists, non-scien¬ 
tists) as well. For both, the likelihood of event occurrence is 
not understood as being uniform throughout a time window. 
Additionally for experts, translation of verbal descriptions 
of event likelihood uncertainty to numerical terms is not 
fully accurate (Doyle, Johnston, McClure, & Paton, 2011; 
Doyle, McClure, Johnston, & Paton, 2014; McClure, Doyle, 
& Velluppillai, 2015). 

7. Uncertainty of data and knowledge about public health 
emergency events influences interactions within and among 
various stakeholders such as groups of experts and between 
experts and policy/ decision makers, which in turn affects 
the decision-making process in complex ways. A final deci¬ 
sion to be communicated to the public (e.g., evacuation 
warning) can be seen as the end point of a chain of decisions 
that includes a flow of uncertainty information where dif¬ 
ferent experts and policy makers in the decision chain un¬ 
derstand uncertainty differently and tend to act in face of 
uncertain information differently. Additionally, uncertain 
scientific knowledge is entwined with values issues (appro¬ 
priate standards for public protection), which makes the 
decision chain process even more complex (Downton et al., 
2005; Morss, 2010; Morss, Demuth, Bostrom, Lazo, & Laz- 
rus, 2015; Ramos, Mathevet, Thielen, & Pappenberger, 
2010 ). 

8. The media are an important stakeholder and if their 
coverage of a public health emergency event emphasizes 
rapidly changing, contradictory, and conflicting informa¬ 


tion, especially that differs from official information from 
authorities, this can increase uncertainty in the public, which 
in turn can lead to several undesirable outcomes in the pub¬ 
lic such as lack of trust in authorities and recommended 
actions; confusion and fear; reduced intentions for health 
protective behaviors; and reduced attention to health risk 
news. Such media coverage also puts a constraint on the 
ability of other stakeholders such as frontline health/medical 
workers to address the public’s uncertainty about effective 
response to the event (Afifi, W., Afifi, T., & Merrill, 2014; 
Quinn, Thomas, & Kumar, 2008; Rousseau et al., 2008; 
Taylor-Clark et al., 2007). 

9. Various stakeholders such as medical/ health care 
workers and policy makers experience uncertainty about a 
public health emergency event. Consequently, the decision 
making regarding communication to the public about the 
event becomes uncertain, when the official information 
about the event is absent or contradictory/ inconsistent. In 
addition, when authorities rush to declare a “fact” about an 
emergent event without transparently acknowledging uncer¬ 
tainties, it can lead to compromised decision making by 
relevant stakeholders (Gesser-Edelsburg, Mordini, James, 
Greco, & Green, 2014; Rousseau et al., 2008). 

3.5.2 Country, Event, Phase, and Vulnerable 
Population Coverage 

As seen from Table 6, the countries covered in the across- 
method synthesized findings showed mostly high and mid¬ 
dle-income countries in Asia, Europe, North America, and 
Oceania. Only one country was covered in Africa and two 
countries in Central and South America. 

The event most covered in the findings was infectious 
disease, in both English language and other UN languages 
studies. Other relatively common events included flood and 
earthquake. All four event phases were covered though there 
was heavy emphasis on the preparation phase, followed by 
onset and containment phases; relatively there was much less 
coverage of the recovery phase and evaluation. 

The findings included only seven studies (four English 
language [Taylor-Clark, Koh, & Viswanath, 2007; Taylor- 
Robinson, Elders, Milton, & Thurston, 2009; Vaughan et 
al., 2012; van Voorst, 2015] and three other UN languages 
[Jakubowski & Charpak, 2004; Masse, Weinstock, Dessy, 
& Moisan, 2011; UN/FAO 2011]) that explicitly examined 
at-risk/ vulnerable populations. Thus, the coverage of such 
populations was minimal. 
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4.0 DISCUSSION 

The synthesis of evidence on the phenomenon of uncer¬ 
tainty during public health emergency events was based on 
findings from 46 studies (33 English language, 13 other UN 
languages). The individual studies were appraised for qual¬ 
ity after which findings were extracted from each study and 
placed within four methodological streams. Next, the indi¬ 
vidual-study findings within each methodological stream 
were synthesized and these findings were evaluated for cer¬ 
tainty/ confidence. Lastly, the within-method synthesized 
findings were further synthesized across the four methods 
to yield a final set of across-method synthesized findings. 
The certainty/ confidence of the final set of synthesized 
findings was judged as moderate (as opposed to high or low) 
based on the aggregated range of method specific assessment 
tool-derived certainty/ confidence appraisals of the within- 
method synthesized findings. 

4.1 Findings from Present Review Vis-a-Vis 
Findings from Existing Reviews 

There were four existing reviews assessed as high and 
moderate quality whose findings were relevant to the present 
review (Bradley, McFarland, & Clarke, 2014; Gesser-Edels- 
burg et al., 2015; Meredith et al., 2008; Visschers, Meertens, 
Passchier, & de Vries, 2009). 4 A review (Liu, Bartz, & Duke, 
2016) was published after the completion of the project is 
included here but was not assessed for quality. The results 
from the present review generally overlap with and extend 
these findings and provide new findings as well. 

The present findings broadly replicate and extend the 
previous findings about the conceptualization of uncer¬ 
tainty and identification of its different aspects. Similar to 
the previous findings, the present review did not find a de¬ 
finitive view of uncertainty in the literature. Although there 
are several conceptualizations of uncertainty, there seems 


to be no consensus on the best view for a public health emer¬ 
gency event. Likewise, although several types and sources 
of uncertainty are noted in the present results, there is no 
clear agreement on a list that would be the most applicable 
to the public health emergency event situation. Along the 
same lines, the present review also notes that the research 
literature remains generally atheoretical. Additionally, the 
present findings also show that the format for presenting the 
likelihood information for occurrence of events in messages 
influences how the information is understood. 

At least three new findings are highlighted in the present 
review. First, the present findings firmly distinguish between 
uncertainty as experience and uncertainty as information. 
Although the two concepts are related — uncertainty infor¬ 
mation can decrease/ increase experience of uncertainty and 
uncertainty experience can modulate how uncertainty in¬ 
formation is interpreted — the findings show that these are 
clearly two separate concepts. Both concepts are essential to 
fully understand the nature of uncertainty in public health 
emergency events. Second, are the findings related to ex¬ 
perts’ decision-making under conditions of uncertainty re¬ 
garding data and knowledge that they have and how this 
uncertainty gets propagated through the chain of decisions 
that lead to a public forecast or warning announcement. 
Third, are the findings related to the role of absent, contra¬ 
dictory, and inconsistent information in the mass media, 
which can increase uncertainty in both the public and med¬ 
ical/ health workers as well as negatively influence the de¬ 
cision-making in organizations and impede their efforts. 

4.2 Suggestions for Practice 

The final set of findings provides an understanding of the 
phenomenon of uncertainty in the situation of public health 
emergency events and the message and activities that can be 
undertaken by authorities to communicate and reduce un¬ 
certainty in this context. Overall, the findings lend them- 


4 An existing review (Vaughan and Tinker, 2008) was rated as low quality and it was “unpacked” for its data-based primary studies, 
which were added to the literature for the present review. The quality of the existing reviews was rated using a modified Assessment of 
Multiple Systematic Reviews (AMSTAR) quality appraisal checklist (Shea et al., 2007). AMSTAR consists of 11 elements that address 
the reviews’ design (i.e., a priori), data extraction, details of the literature search, inclusion of grey literature, characteristics, methods, 
and scientific quality of included studies, publication bias, and acknowledgement of conflict of interest(s). Each area in AMSTAR is 
assessed using “yes,” “no”, “can’t answer,” or “not applicable.” Studies received a final rating of “high” (no significant flaws), “moderate” 
(minor flaws impacting credibility/validity), or “low” (some flaws likely to impact credibility/ validity). Two coders did the coding in¬ 
dependently with high agreement. The final quality assessment was judged after the coders resolved any differences. 
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selves to at least four suggestions for authorities when 
designing messages for addressing uncertainty related to 
public health emergency events. 

First, authorities should keep in mind that uncertainty is 
both a lived experience and a characteristic of information, 
and in both senses is multifaceted with multiple types and 
sources (e.g., Afifi, W., Felix, & Afifi, T., 2012; Alipour et 
al., 2015). This means that authorities need to design mes¬ 
sages with a high level of specificity of uncertainty type and 
source, where the magnitude of uncertainty may vary con¬ 
siderably across the types and sources as well as across the 
phases of an event. This also means that before and while 
designing messages, authorities need to monitor the public’s 
experience of uncertainty due to the actual emergency event 
and the public’s response to uncertainty information in mes¬ 
sages about the event as these constitute two different, 
though overlapping, aspects that together determine deci¬ 
sion-making and behavioral response. 

Second, there is general agreement among experts and 
researchers, though with some caveats, that communication 
by authorities to the public should include explicit informa¬ 
tion about uncertainties associated with events (e.g., Aldunce 
& Leon, 2007; Frewer et al., 2003; Karan, Aileen, & Elaine, 
2007). It is important to ensure that the information pro¬ 
vided is consistent and not contradictory, and is presented 
clearly and in an easy to understand manner. Messages 
conveying uncertainty information that disregard this will 
fail to work. 

Third, authorities should be sensitive to that all vulner¬ 
able groups may not process uncertainty information the 
same way (e.g., Vaughan et al., 2012). The findings note, for 
example, the processing and effects differences between 
urban African American and Hispanic minorities in the 
United States with regards to uncertainty information in 
messages. In this regard, authorities should also be cognizant 
of the fact that the life circumstances of vulnerable groups, 
such as people from low socioeconomic backgrounds, might 
have a myriad of uncertainties stemming from poverty, 
chronic illness, among other factors (e.g., van Voorst, 2015). 
As such, the uncertainty associated with a public health 
emergency event might be just one source of uncertainty 
among many others. 

Fourth, authorities should remain keenly aware that ex¬ 
perts’ (both scientists and non-scientists) and policy makers’ 
handling of uncertainty information in forecasting, warning, 
and other similar decision-making is a complex process that 


may be similar to the general public’s processing of uncer¬ 
tainty information (e.g., Downton et al., 2005; Gesser-Edels- 
burg et al., 2014)). Along the same lines, how media 
professionals as well as health and rescue workers experience 
uncertainty and process information about it may not be too 
different from the public’s (e.g., Rousseau et al., 2008). As 
such, authorities must not solely focus on communicating 
uncertainty information to the public but also keep in mind 
the need for addressing uncertainty for various other stake¬ 
holders. 

Authorities should carefully consider at least these four 
aspects of uncertainty in the context of public health emer¬ 
gency events when developing the best ways to communicate 
uncertainties to the general public, at-risk/ vulnerable popu¬ 
lations, and stakeholders. 

4.3 Research Gaps in the Reviewed Literature 

The present review identified seven main gaps in the 
reviewed literature on the phenomenon of uncertainty dur¬ 
ing public health emergency events. These gaps suggest av¬ 
enues for further research. First, there should be a 
comprehensive examination of the various conceptualiza¬ 
tions and sources of uncertainty, separately as well as joint¬ 
ly for uncertainty experience and uncertainty information. 
There are studies that investigated different sets of sources, 
but the review did not identify any study that comprehen¬ 
sively examined all relevant concepts and sources, and test¬ 
ed their relationships with outcome and predictor variables 
of interest. 

Second, there is a paucity of studies examining message 
designs such as linguistic choices and visual formats that 
can augment understanding of uncertainty information. 
There do exist studies that have investigated this, but the vast 
majority have been conducted with college students in labo¬ 
ratories. Such studies in the future need to be conducted in 
the field with populations affected or likely to be affected by 
public health emergency events. 

Third, there is insufficient comparative research across 
countries, especially across low and high income countries. 
To fully understand how the characteristics of low income 
countries, especially in terms of infrastructure, history, and 
political climate, might influence uncertainty processes dif¬ 
ferently relative to high income countries, there needs to be 
comparison of such countries. If practices of health author¬ 
ities need to be different across low and high income coun- 
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tries, specific factors and relationships among the factors 
that contribute to uncertainty processes should be compared 
across countries. 

Fourth, there is not enough attention paid to the most 
vulnerable and disadvantaged populations. These are often 
the populations who have the least access to information 
resources and exposure to official information before, dur¬ 
ing, and after an event, and as such face the most uncer¬ 
tainty. Uncertainty is alleviated through information and 
when there is insufficient access to traditional and new 
media sources, information may be predominantly sought 
from interpersonal networks. There are not enough studies 
that investigate information seeking processes in such media 
access-poor populations. 

Fifth, completely absent in the literature are longitudinal 
studies. It is not always necessary to have randomized com¬ 
parison group research design, which may be precluded due 
to the nature of public health emergency events, to draw out 
causal relationships. Such linkages between variables of 
interest, such as health protection behavior as an outcome 
of uncertainty information in a message, can also be exam¬ 
ined using a longitudinal research design where data of inter¬ 
est are measured at multiple time points. Such a research 
design can better reveal how uncertainty dynamically varies 
during the phases of an event; even if say, preparation and 
recovery phases are only used for data collection, this will 
still provide insight into how uncertainty information affects 
different variables across the phases. Such a design can also 
provide knowledge about how uncertainty experience varies 
and how it interacts with uncertainty information through 
the course of an event. 

Sixth, there is insufficient research on how uncertainty 
information is determined and processed by experts and then 
disseminated to their colleagues within and across organiza¬ 
tions and then to the public. Although some research exists, 
it has not been conducted for decision-making processes in 
a wide variety of public health emergency events, organiza¬ 
tions, and scientific/ technical areas and disciplines. It is 
commonly assumed that all experts in all domains accu¬ 
rately understand and in turn correctly disseminate uncer¬ 
tainty information. The sample of studies addressing this 
topic in the present review, though small in number, suggests 
otherwise. 

Seventh, theories of communication research that di¬ 
rectly speak to uncertainty have not been used to investigate 
this phenomenon in the context of public health emergency 


events. There are at least three theories, as discussed by 
Bradac (2001), that can substantially enhance the under¬ 
standing of uncertainty experience and processing of uncer¬ 
tainty information. Uncertainty reduction theory (Berger & 
Calabrese, 1975) proposes that uncertainty experience is an 
aversive mental state that motivates seeking of information 
to reduce the uncertainty. In contrast, uncertainty management 
theory (Babrow, Hines, & Kasch, 2000; Brashers, 2001) pos¬ 
its that uncertainty experience is not necessarily a negative 
mental state that requires reduction, but can be experienced 
as positive or neutral as well such that it may motivate infor¬ 
mation seeking to increase rather than reduce uncertainty. 
Extending this view that uncertainty experience may be both 
negative and positive, problematic integration theory (Babrow, 
2001; Brashers, 2001) posits that uncertainty is linked to as¬ 
sessments of probability of an outcome and favorability of 
the outcome, and their integration with one another and with 
existing knowledge and beliefs. Public health emergency 
communication research focuses on the processing of prob¬ 
ability aspects of uncertainty information. Future research 
should also investigate the perception of favorability of un¬ 
certainty information to see its role in the experience of 
uncertainty during public health emergency events. The 
conditions of threat that constitute such events are likely to 
result in an experience of uncertainty that is aversive along 
with a desire to reduce this state; however, the last two theo¬ 
ries above imply that this assumption may not perhaps hold 
true for all populations, all phases of an event, or all types 
of uncertainty information. Future research should measure 
uncertainty experience in both its negative and positive 
forms (see Morss & Hayden, 2010 in this regard) to see how 
the degree of this aversive state might influence the type and 
amount of information that is sought from authorities during 
public health emergency events. 

4.4 Implications for Theory 

The present review showed a general absence of testing 
and development of theories and models. To develop effec¬ 
tive strategies for addressing uncertainty during public health 
emergency events, integrative models and theoretical frame¬ 
works that rest on empirical findings can increase the likeli¬ 
hood of accurate predictions, which can assist with planning 
and management of such events. Based on the final set of 
findings from the present review, we offer three propositions 
that can contribute to theoretical frameworks for understand- 
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ing uncertainty in the context of public health emergency 
events. 

First, a distinction between uncertainty experience and 
uncertainty information is required. The two concepts are 
intertwined and to fully understand the nature of uncer¬ 
tainty, it is not advisable to investigate one without the oth¬ 
er. The two concepts, their various sources, and the 
relationships among them quite likely behave similarly, if 
not identically, in the general public and in communities of 
experts, policy makers, health and rescue workers and offi¬ 
cials, and other related stakeholders. That is, it is quite like¬ 
ly that uncertainty is experienced in the same general ways 
by the public and other stakeholders, and uncertainty infor¬ 
mation is understood and misunderstood by the general 
public and other stakeholders alike. 

Second, the experience of uncertainty may be a defining 
feature of a public health emergency event not only for the 
public and experts, policy makers, and related stakeholders, 
but for journalists and other media professionals as well. The 
mandate of media professionals is to report all relevant in¬ 
formation to the public in a timely manner, including infor¬ 
mation that might be contradictory/ inconsistent, which will 
be perceived as uncertain. This media information is re¬ 
ceived by and influences the public as well as frontline med¬ 
ical/ health workers and other related stakeholders who 
access the media during public health emergency events. 

Third, given the first two propositions, perhaps cau¬ 
tiously, a theoretical proposition in the form of a metaphor 
of a cascade of uncertainty propagation can be forwarded. 
The propagation of uncertainty, both experience and infor¬ 
mation, can be seen as having multiple origination points in 
experts, public, policy makers, media professionals, frontline 
medical/health officials and workers, and other stakeholders 
as soon as they detect, specify, or experience a public health 
emergency event and begin disseminating uncertainty infor¬ 
mation about the event. The uncertainty cascade does not 
only include the decision to disseminate a forecast or warn¬ 
ing message by the authorities but it also includes the deci¬ 
sions the public, media, and other stakeholders make after 
processing uncertainty information under experienced un¬ 
certainty. The uncertainty cascade travels from multiple 
points to multiple points (e.g., from media to public, from 
media to frontline health workers, from public to policy 
makers, from policy makers to media) in a reciprocal 
exchange as organizations, authorities, public, and other 
stakeholders make new decisions in response to the decisions 


made by others. This system-level view of uncertainty in 
public health emergency events might assist authorities in 
constructing messages that communicate uncertainty with 
greater specificity and nuance in relation to the unique cir¬ 
cumstances of the different intended audiences and also plan 
for how unintended audiences might be affected by the same 
information. 

None of the theories and models commonly used to study 
public health emergency events directly focus on uncer¬ 
tainty. The final set of findings and above propositions can 
deepen the contributions of existing theories by suggesting 
pathways for explicitly incorporating uncertainty in their 
formulations. 

Sheppard, Janoske, and Liu (2012; see Figure on p. 3) 
provide an overview of a wide range of theories and models, 
both at the individual-psychological and organizational lev¬ 
els, that have been used in or have relevance to the study of 
public health emergency events. None of the theories and 
models have uncertainty as a prominent variable, and so they 
can include uncertainty experience and uncertainty informa¬ 
tion as distinct but interconnected variables. The theories 
and models typically investigate the general public; they can 
be extended to studying risk perceptions and processing in 
experts, policy makers, health/medical staff, and related 
stakeholders as well as media professionals. These same 
suggestions can also be made for the various individual- 
level psychological models that examine risk perceptions (for 
an overview see Covello, Peters, Wojtecki, & Hyde, 2001; 
see also Glik, 2007). These models explain how risk informa¬ 
tion is processed, how risk perceptions are formed, and how 
risk decisions are made, which all can be informed by giving 
uncertainty a salient role and studied in populations other 
than the general public. 

Chaos theory (Seeger, 2002; Sellnow, Seeger, &Ulmer, 
2002) and dynamic systems (Burns & Slovic, 2007) perspec¬ 
tives on public health emergency events are in most concor¬ 
dance with the cascade of uncertainty propagation 
proposition suggested above. The perspectives seek to ad¬ 
vance understanding of public health emergency events by 
viewing then as systems that are nonlinear, complex, and 
unpredictable. These perspectives can easily incorporate in 
their theorizing and computer modeling how uncertainty 
experience and uncertainty information diffuses through the 
public and various stakeholders that together constitute a 
community. 
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4.5 Limitations of the Present Review 4.6 Conclusion 


The present systematic review has two main limitations. 
First, the non-English UN languages articles and reports 
were only selectively, and not fully, translated into English, 
which may have led to some information to be missed during 
data extraction. Second, the data extraction from individu¬ 
al studies was done principally by one person (English lan¬ 
guage by the lead author of the review; other UN languages 
by a native reader) as was also the case for the synthesis of 
findings across studies (by the lead author of the review), 
with the results vetted by at least one other member of the 
research team. However, this cross-checking process was not 
formalized, which prevented the statistical calculation of 
inter-coder reliability to determine the degree of consistency 
of results. 


Uncertainty experience and communication related to it 
is a complex phenomenon that is inherent to a public health 
emergency event. The final set of synthesized findings from 
the present systematic review deepen our understanding of 
the phenomenon relative to previous reviews and lead to 
suggestions for practice, future research, and a theoretical 
proposition that can guide development of a conceptual 
model. As a whole, these can assist various stakeholders with 
managing uncertainty experience and disseminating uncer¬ 
tainty information to reduce negative health outcomes and 
enhance recovery efforts. 
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Search Terms (back to text) 


Main Search Term 

Boolean ‘And’ Term 

(Used With Any of the Main Search Terms) 

Disaster* 

At risk population; at risk community 

Disaster plan* 

At risk 

Communication 

Uncertaint* 

Risk communication 

Warning 

Emergenc* 

Community participation 

Hazard* 

Stakeholder 

Risk* 

Social network communit* 

Threat* 

Public; audience 

Emergency preparedness 

Duty to protect 

Emergency management 

Safety 

Crisis communication 

Crisis (or other truncation for a specific database, e.g. ?, #) 

Risk population; risk community 

Disaster preparedness 

Community 

Hazard communication 

Social media 

Emergency communication 

Facebook 

Catastrophe communication 

Twitter 

Health communication 

New media 


Table 2. Study Selection (back to text) 


Language 

Total Number of 

Titles and 

Abstracts Scanned 

Total Number of 

Full-Texts 

Quickly Scanned 

Total Number of 

Full-Texts 

Downloaded 

Total Number of 

Full-Texts Fully 

Read 

Total Number of Full- 

Texts Selected for Data 

Extraction (Only Data- 
Based Primary Studies) 

English 

2909 

1706 

73 

73 

33 

Arabic 

6720 

— 

57 

0 

0 

Chinese 

800 

— 

125 

2 

1 

French 

196 

— 

78 

5 

5 

Russian 

870 

— 

639 

3 

3 

Spanish 

No accurate 

data 

— 

No accurate 

data 

6 

4 
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Table 3. Characteristics of Studies (back to text) 


Relevancy Method Country Focus Disaster/ Emergency Type Emergency Phase 

General 


At-risk Groups 


QN-CG: 1 Australia: 1 
QN-DS: 10 Canada: 2 
QL: 11 Chile: 1 
MM, CS: 11 China: 1 

Europe general: 2 
France: 1 
India: 1 
Indonesia: 1 
Iran: 1 
Israel: 1 
Japan: 1 
New Zealand: 1 
Singapore: 1 
South Africa: 1 
Thailand: 1 
United Kingdom: 2 
United States: 12 


Direct: 6 
Indirect: 17 
Partial: 10 
Unclear: 0 


Direct: 7 
Indirect: 6 


QN-CS: 0 
QN-DS: 2 
QL: 4 

MM, CS: 7 


Austria: 1 
Belgium: 1 
Canada: 3 
Chile: 1 


English Language (33 Studies) 


General: 1 
Bioterrorism: 2 
Cyclones/ Hurricanes: 3 
Earthquake: 5 
Flood: 7 

Food Contamination: 2 
Industrial Accident: 2 
Infectious Disease: 6 
Landslides: 1 
Tornado: 1 
Volcanic: 2 
Wildfire: 1 


All Phases: 5 
Preparation: 16 
Onset: 7 
Containment: 5 
Recovery: 4 
Evaluation: 1 


Other UN Languages (13 Studies) 
General: 4 Preparation: 2 

Chemical/Air Pollution: 1 Onset: 1 

Flooding: 1 Containment: 1 

Food Safety: 1 Evaluation: 2 


Yes: 4 

[Low socio-economic status (SES): 2, 
Minorities: 2, 

School Children: 1] 


Yes: 3 

[Children: 2 
Chronic Disease: 1 
Low-SES: 1 
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China: 1 Infectious Diseases: 6 

Finland: 1 

France: 1 

Mexico: 1 

Norway: 1 

Russia: 3 

Spain: 1 

United Kingdom: 1 
General: 1 


Preparation, & Containment: 1 
Preparation, & Evaluation: 1 
Preparation, Onset, & Containment: 1 
Preparation, Onset, & Recovery: 1 
Preparation, Onset, & Evaluation: 1 
Onset, Containment, & Evaluation: 1 


Minorities: 1 
Older People: 1 
Pregnant Women: 1] 


Notes, (back to text) 

Some categories are not mutually exclusive and so the frequencies will not sum to the total of 33 (English language) and 13 (Other UN languages). 
Method: Quantitative-Comparison Groups (QN-CG); Quantitative-Descriptive Survey (QN-DS); Qualitative (QL); Mixed-Method/Case Study (MM, CS) 


Table 4. Quality Appraisal of English Language Individual Data-based Primary Studies (back to text) 


Method 

Citations (first author only, unless noted otherwise) 


Quality Appraisal Rating 

QN-CG 

Johnson (2015) 

Moderate 


QN-DS 

Afifi (2012); Sharma (2012); Spence (2007); Vaughan (2012) 

Moderate 


QN-DS 

Burke (2009); Doyle (2011); Doyle (2014); Janmaimool (2014); McClure (2015); Miles (2003) 

Weak 


QL 

Alipour (2015); Morss (2015); Taylor-Clark (2007) 

High 


QL 

Acar (2011); Afifi (2014); Bird (2012); Holmes (2009); Morss & Hayden (2010); Pappenberger 
(2013); Quinn (2008); Taylor-Robinson (2009) 

Moderate 


MM, CS 

Dabner (2012); Downton (2005); Frewer (2003); Karan (2007); Morss (2010); Ramos (2010) 

High 


MM, CS 

Aldunce (2007); Gesser-Edelsburg (2014); Rousseau (2008); van Voorst (2015) 

Moderate 


MM, CS 

Skinner (2014) 

Low 



Notes. 

Method: Quantitative-Comparison Groups (QN-CG); Quantitative-Descriptive Survey (QN-DS); Qualitative (QL); Mixed-Method/ Case Study (MM, CS) 

Quality: QN-CG - High, Moderate, Low, Very low; QN-DS - Strong, Moderate, Weak; QL - High, Moderate, Low, Very low; MM, CS - High, Moderate, Low, Very low. 
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Table 5. Synthesis of Findings Within Methodological Stream and Evaluation of Certainty / Confidence (back to text) 


Citations (first author only, 

Method Synthesized Finding Statement (with subgroup analysis of type, phase, and unless noted otherwise) Sup- 
country of disaster, and vulnerable population) porting Synthesized Finding 

Within Method Stream 


QN-CG In the United States for an infectious disease event for onset and containment 
phases, trust in authorities may show a slight decrease as a result of openly 
acknowledging uncertainties in messages. Flowever, this decrease is only for 
a small proportion of the total number of message recipients; for the vast 
majority of message recipients, there is no change in their level of trust. 

QN-DS In Thailand, New Zealand, the United Kingdom, and the United States, for 
food contamination, industrial accident, volcanic, and wildfire events, and 
for preparation and recovery phases, it should be noted that there are differ¬ 
ent types/components to the public’s experience of uncertainty. As examples, 
one classification notes three types: uncertainty regarding personal safety; 
safety of home; and safety of close others. Another classification also notes 
three types: uncertainty about event knowledge; data; and outcome. Along 
the same lines, there is risk assessment uncertainty and event outcome un¬ 
certainty. Another classification shows seven types of uncertainty: uncer¬ 
tainty about who is affected; temporal uncertainty (uncertainty about past 
and future states); measurement uncertainty; uncertainty due to scientific 
disagreement; uncertainty about the risk to humans; uncertainty about the 
extent (or size) of the risk; and uncertainty about how to deal with and reduce 
the risk. 


Johnson (2015) 


Afifi (2012); Janmaimool 
(2014); Doyle (2011); Miles 
(2003) 


Evaluation of 
Certainty/ 
Confidence of 
Synthesized 
Finding Within 
Method Streams 
Low to Moderate 


Low 


Explanation of Evaluation 


Two studies reported in article, 
one not a randomized group 
comparison. Some evaluation 
categories not applicable or 
‘cannot tell’. 

Not fully overlapping findings 
by 4 studies, individually ap¬ 
praised as moderate (1) and 
weak (3). 
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QN-DS 


In China, Mexico, and the United States, for earthquake, hurricane, infec¬ 
tious disease, and wildfire events, and for all phases, the public’s experience 
of uncertainty was due to lack of information and uncertainty was reduced 
by greater information, and the public actively sought out information to 
reduce its uncertainty. Uncertainty in a time of crisis can motivate individu¬ 
als to engage in information seeking, which can alleviate the uncertainty. 
People seek both general and specific information, and there here are demo¬ 
graphic and mass medium differences in information seeking. People seek 
information (and coping support) from personal networks as well to reduce 
uncertainty and its impact on mental health. 

QN-DS In Thailand, New Zealand, and globally, for foodborne illness, industrial 
accident, and volcanic events, for preparation and containment phases, and 
including for low SES population, there is general agreement among experts, 
both scientists and non-scientists, and researchers that communication by 
authorities to the public should include explicit information about uncertain¬ 
ties associated with events. 

QN-DS In India, Thailand, New Zealand, the United Kingdom, and the United 
States, for bioterrorism, cyclone, earthquake, food contamination, and in¬ 
dustrial accident events, for preparation, onset, and containment phases, 
and including for urban minority African American and Hispanic popula¬ 
tions, knowledge and understanding of uncertainty information provided in 
messages as predictor is associated with outcomes of: trust and confidence 
in authorities; perception of transparency of authorities; experience of fear; 
response to warnings; likelihood of preparation; and risk perceptions. 
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Afifi (2012); Burke (2009); High 
Spence (2007); Muniz (2011) 

SP 


Overlapping findings by 4 stud¬ 
ies, individually appraised as 
strong (1), moderate (2), and 
weak (1). 


Doyle (2011); Janmaimool Moderate 
(2014); FAO/WHO (2011) FR 


Overlapping findings by 3 stud¬ 
ies, individually appraised as 
moderate (1) and weak (2). 


Janmaimool (2014); Sharma Moderate 
(2012); McClure (2015); Miles 
(2003); Vaughan (2012) 


Overlapping findings by 5 stud¬ 
ies, individually appraised as 
moderate (2) and weak (3). 
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QN-DS In New Zealand, for earthquake event, and for preparation phase, the public’s 
understanding of uncertainty information provided in messages was error 
prone. People rated the likelihood of event occurrence higher in later intervals 
than in earlier intervals of a time window. This pattern was observed despite 
the fact that the verbal descriptions of the likelihood of event occurrence in 
a specific time window did not imply that the event likelihood changed across 
different intervals of the time window. 

QN-DS In New Zealand, for volcanic events, and for preparation phase, experts’ 
(both scientists, non-scientists) understanding of uncertainty information 
provided in messages was error prone. Experts rated the likelihood of event 
occurrence higher in later intervals than in earlier intervals of a time window. 
Experts did not view the likelihood of a volcanic eruption as being uniform 
throughout a time window; they instead viewed the likelihood of an event 
in an earlier time interval as being lower and in a later time interval as being 
higher as opposed to uniform across all periods in the time window. Simi¬ 
larly, attempts to accurately translate verbal descriptions of event likelihoods/ 
uncertainty to numerical terms were with problems. 

QL In France, Iran, and the United States, for earthquake, flood, and tornado 

events, for preparation and recovery phases as well as for evaluation, there 
are different types/ components to the public’s experience of uncertainty. 
Experience of uncertainty is related to: disruption of roles and responsibili¬ 
ties; employment; influx of non-local population; reconstruction without 
considering of local culture; and not understanding the causes of an event. 
Experience of uncertainty changes across the time course of an event that 
includes uncertainty regarding: impact of event; future of schools and vil¬ 
lage/ town; and decisions about rebuilding. 
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McClure (2015) 


Low 


Finding based on one study, 
appraised as weak. 


Doyle (2011); Doyle (2014) Low 


Finding based on two studies, 
both appraised as weak. 


Overlapping findings by 3 stud¬ 
ies, individually appraised as 
high (1), moderate (1), and low 
( 1 ) 


Afifi (2014); Alipour (2015); Moderate 
Duchene (2004) FR 


QL 


In Australia, Japan, the United Kingdom, and the United States, for earth¬ 
quake, flood, tornado, and infectious disease, for all phases, and including 
for school children, lack of information increases experience of uncertainty 
and information from authorities reduces this uncertainty. Authorities must 
provide information speedily, timely, and reliably. People actively seek in¬ 
formation to reduce their uncertainty, especially through social media, and 
the authorities too should use this medium for information dissemination. 

QL In the United States, for bioterrorism, general public health, and tornado 

events, for all four phases, and including for low SES minorities population, 
contradictory and inconsistent information in the media may be seen as 
uncertain information, which leads to several outcomes including: experience 
of uncertainty; lack of trust in authorities and recommended actions; confu¬ 
sion and fear; reduced intentions for health protective behaviors such as 
vaccination; and reduced attention to health risk news. 

QL In Canada, China, several European countries, and the United States, for 

bioterrorism, flood, general public health, and infectious disease, for prepa¬ 
ration, onset, and containment phases as well as evaluation, and including 
for low SES minorities, pregnant women, children, and people with chronic 
disease populations, there is general agreement among experts and research¬ 
ers that communication by authorities to the public should include explicit 
information about uncertainties associated with events. It is important to 
ensure that the information provided is consistent and not contradictory, 
and is presented clearly and in an easy to understand manner. 
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Acar (2011); Afifi (2014); Bird 
(2012); Taylor-Robinson 
(2009) 

High 

Overlapping findings by 4 stud¬ 
ies, all individually appraised 

as moderate. 

Afifi (2014); Quinn (2008); 
Taylor-Clark (2007) 

Moderate 

Overlapping findings by 3 stud¬ 
ies, individually appraised as 
high (1) and moderate (2). 

Holmes (2009); Pappenberger 
(2013); Quinn (2008); Taylor- 
Clark (2007); Masse (2011) 
FR; Li (2010) CH 

Moderate 

Overlapping findings by 6 stud¬ 
ies, individually appraised as 
high (2), moderate (3), and low 
(1). 
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QL In France, Russia, and the United States, for flood, hurricane, and general 

public health events, and for preparation and onset phases as well as evalu¬ 
ation, uncertainty information provided in messages as predictor is associ¬ 
ated with the outcomes of: confidence in forecasts; reduction in circulation 
of misinformation; and improved risk management. However, the uncer¬ 
tainty about specific parameters of an event may sometimes leave not have 
enough time to prepare property or move belongings to a safe location. Also, 
the phrasing of the uncertainty information may sometimes be interpreted 
negatively, which may affect response to future risk communication. 

QL In the United States, for flood event, and for onset phase, how experts make 

decisions about forecasting and warnings under conditions of uncertain data 
is a complex process. Experts have to use (scientifically) uncertain data, and 
in rapidly evolving situations where multiple actors have to make interre¬ 
lated decisions under uncertainty, there is a greater danger of risk assessment 
propagating across individuals in unintended ways. Although uncertainty 
can be reduced by actively seeking and obtaining data from multiple sourc¬ 
es, there is a need to improve experts’ decision-making under conditions of 
uncertain data in the context of their interactions with others. 

MM, CS In Russia and the United States, for flood and general public health events, 
and for preparation phase, there are several types of uncertainty information 
that can be put in messages by authorities. In particular, these include knowl¬ 
edge uncertainty (limitations of scientific understanding of complex natural 
processes and future changes) and sampling uncertainty (uncertainty in 
estimates calculated using limited data samples from naturally variable 
processes). The uncertainties can also be about results of checks and ex¬ 
aminations of event control mechanisms and health affecting properties of 
dangerous materials produced by industry. It should be noted that often 
uncertainty becomes confounded with values issues, which deal with the 
appropriate standards of public protection. 
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Morss & Hayden (2010); Moderate 
Duchene (2004) FR; Gryzu¬ 
nova (2012) RU 


Overlapping findings by 3 stud¬ 
ies, individually appraised as 
high (1), moderate (1), and low 
( 1 ) 


Morss (2015) 


Moderate Finding based on one study, 

appraised as high. 


Downton (2005); Podkorytov Moderate 
(2014) RU 


Overlapping findings by 2 stud¬ 
ies, both individually appraised 
as high. 


MM, CS 


MM, CS 


MM, CS 


In Chile and Indonesia, for flood and landslide events, for preparation phase, 
and including for low SES people, for authorities to develop messages that 
contain uncertainty information, it is important to keep in mind the whole 
living environment of the intended audience, which may be full of uncertain¬ 
ties due to poverty. It should be recognized that issues of development and 
environment are just as central to reduced uncertainty as messages from 
authorities. 

In Chile, Finland, Singapore, South Africa, and New Zealand, for air pol¬ 
lution, earthquake, infectious disease, industrial accident, landslide, and 
general public health events, for all phases as well as evaluation, uncer¬ 
tainty experienced by the public as an outcome is associated with the follow¬ 
ing predictors: disaster management, coordination, and cooperation at the 
local level; disaster management plan that is interactive with the public, and 
that includes all mass and social media; integrated management of official 
response to event across all mass and social media and other domains; 
regular and timely information, including via social media; detailed informa¬ 
tion disseminated, including through personal networks (doctors, nurses, 
community leaders); regular updates about the event progression through 
the mass media; information about the probability and consequences of 
events; and differing levels of risk perceptions. 

In Canada and France for an infectious disease event, and for onset and 
containment phases, uncertainty about an event conveyed by mass media 
coverage through rapidly changing, contradictory, and conflicting informa¬ 
tion, especially that differs from official information from authorities, in¬ 
creases uncertainty and fear in the public, and puts a constraint on health/ 
medical workers ability to address the public’s uncertainty. 


99 


Communicating Uncertainty During Public Health Emergency Events 


Aldunce (2007); Voorst (2015) Moderate Overlapping findings by 2 stud¬ 

ies, both individually appraised 
as moderate. 


Aldunce (2007); Dabner Moderate 
(2012); Karan (2007); Skinner 
(2014); Barengo (2011) SP; 

Francescutti (2007) SP; Valle- 
jos-Romero (2013) SP 


Overlapping findings by 7 stud¬ 
ies, individually appraised as 
high (4), moderate (1), and low 
( 2 ) 


Rousseau (2008) 


Moderate Finding based on one study, 

appraised as moderate. 
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MM, CS 


MM, CS 


MM, CS 


In Canada, France, and Israel, for infectious disease event, and for onset and 
containment phases, and for medical/ health care workers and policy mak¬ 
ers. As a result of absent or contradictory and inconsistent information from 
authorities, medical/ health care workers and policy makers experience 
uncertainty and the organizational decision making regarding communica¬ 
tion to the public becomes uncertain. Instead of providing transparent com¬ 
munication regarding the uncertainty surrounding an emergent event, if 
authorities rush to declare a “fact” about the event without adequate infor¬ 
mation, it can lead to compromised decision making and efforts by organiza¬ 
tions. 

In Austria, Belgium, Canada, Norway, Russia, South Africa, the United 
Kingdom, and several European countries, for flood, infectious disease, 
industrial accident, and general public health, and for all four phases as well 
as evaluation, uncertainty information in messages provided by authorities 
as predictor is associated with the following outcomes: reduced experienced 
uncertainty; reduced uncertainty about protection actions; avoidance of 
information void; reduced misinformation; prevention of rumors; reduced 
indifference; reduced reliance on sensationalized stories; increased sense 
that situation is under control; and efficiency, quality, and value of forecasts. 
The uncertainty information should be timely, full, and unbiased. 

In the United States, for food contamination event, and for preparation phase, 
experts/scientists indicate that providing information about scientific un¬ 
certainty will have a negative impact on the extent to which the public trusts 
science, scientists, and scientific institutions; their view is that the general 
public is unable to conceptualize uncertainties associated with risk manage¬ 
ment processes and so providing the public with information about uncer¬ 
tainty will cause panic and confusion regarding the extent and impact of a 
particular event. 
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Gesser-Edelsburg (2014); Moderate 
Rousseau (2008) 


Overlapping findings by 2 stud¬ 
ies, both individually appraised 
as moderate. 


Ramos (2010); Skinner (2014); Moderate 
Jakubowski (2004) FR; Lord 
(2009) FR; Pozdnyakov (2011) 

RU 


Overlapping findings by 5 stud¬ 
ies, individually appraised as 
high (4) and low (1). 


Frewer (2003) 


Moderate 


Finding based on one study, 
appraised as high. 
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Downton (2005); Morss (2010); High Overlapping findings by 3 

Ramos (2010) studies, all individually apprai¬ 

sed as high. 

des a flow of uncertainty information. It is also important to assess how 
different people in the decision chain perceive and understand uncertainty, and 
tend to act in face of uncertain information. For example, policy/ decision 
makers may not fully understand scientific uncertainty and may default to 
their intuitions and experience to make decisions. Additionally, uncertain 
scientific knowledge is entwined with values issues (appropriate standards for 
public protection), which makes the decision chain process more complex. 


MM, CS In several European countries and the United States, for flood events, and for 
preparation phase as well as evaluation, uncertainty of data and knowledge 
influences decision making and interactions within and among groups of 
experts and between experts and policy/ decision makers. It is important to 
see a final policy decision as the end point of a chain of decisions that inclu- 


Notes. (back to text) 

Method: Quantitative-Comparison Groups (QN-CG); Quantitative-Descriptive Survey (QN-DS); Qualitative (QL); Mixed-Method/ Case Study (MM, CS) 

Citations-Language: English has no suffix; Arabic (AR); Chinese (CH); French (FR); Russian (RU); Spanish (SP) 

Certainty/ Confidence Evaluation: QN-CG (GRADE) - High; Moderate; Low; Very low; QN-DS (GRADE Adapted) - High; Moderate; Low; Very low; QL (CERQual) - High; 
Moderate; Low; Very low; MM, CS (as appropriate) - High; Moderate; Low; Very low 
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Table 6. Synthesis of Findings Across Methodological Streams (back to text p. 12, p. 14) 


Synthesized Finding Across Method Streams (with subgroup analysis of type, phase, and country Citations (first author only, unless noted 

of disaster, and vulnerable population) otherwise) Supporting Synthesized Evaluation of Certainty/ Confidence 1 

Finding Across Method Stream 


There are several different types/ sources of uncertainty associated with an event, related to both 
uncertainties experienced by the public and also uncertainty information than can be put in 
messages, that authorities need to keep in mind. As some examples, uncertainty can be regarding: 
safety of person, home, and close others; event knowledge, sampling, data, and outcome; and 
influx of non-local people. Experience of uncertainty changes across the time course of an event 
that includes, for example, uncertainty regarding impact of event (onset, containment) and future 
of schools and village/ town and decisions about rebuilding (containment, recovery). It should be 
noted that, particularly for authorities, uncertainty can become confounded with values issues, 
which deal with the appropriate standards of public protection. The countries covered include 
France, Iran, New Zealand, Russia, Thailand, the United Kingdom, and the United States. 

Events include earthquake, flood, food contamination, industrial accident, tornado, volcanic, 
wildfire, and general public health. Preparation and recovery phases are covered along with 
evaluation. No vulnerable populations are included. 


Afifi (2012); Afifi (2014); Alipour 
(2015); Downton (2005); Doyle (2011); 
Janmaimool (2014); Miles (2003); 
Duchene (2004) FR; Podkorytov (2014) 
RU 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): Low to 
High 

QL (CERQual): Moderate to High 
MM, CS: Moderate to High 
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The public’s experience of uncertainty is due to lack of information. Uncertainty is reduced by 
greater information and the public actively seeks out information to reduce its uncertainty. Thus, 
information from authorities can reduce the public’s uncertainty. People seek both general and 
specific information, and there are demographic and mass medium differences in information 
seeking; people also seek information (and coping support) from personal networks to reduce 
uncertainty and its impact on mental health. The uncertainty experienced by the public as an 
outcome is associated with other predictors as well, that, as examples, include: disaster manage¬ 
ment, coordination, and cooperation at the local level; disaster management plan that is interacti¬ 
ve with the public, and that includes all mass and social media; integrated management of official 
response to event across all mass and social media and other domains; speedy, regular, timely 
reliable, and detailed information about event progression, including via social media and 
personal networks (doctors, nurses, community leaders); regular updates about the event progres¬ 
sion through the mass media; and differing levels of risk perceptions. The countries covered inclu¬ 
de Australia, Chile, China, Finland, Japan, Mexico, New Zealand, Singapore, South Africa, the 
United Kingdom, and the United States. Events include air pollution, earthquake, floods, hurrica¬ 
ne, industrial accident, infectious disease, landslide, tornado, wildfire and general public health. 
All four phases of an event are covered along with evaluation. School children as vulnerable 
populations included. 


Acar (2011); Afifi (2012); Afifi (2014); 
Aldunce (2007); Bird (2012); Burke 
(2009); Dabner (2012); Karan (2007); 
Skinner (2014); Spence (2007); Taylor- 
Robinson (2009);Barengo (2011) SP; 
Francescutti (2007) SP; Muniz (2011) 
SP; Vallejos-Romero (2013) SP 


QN-CG (GRADE): — 


QN-DS (GRADE Adapted): Low to 
High 

QL (CERQual): Moderate to High 
MM, CS: Moderate 
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There is general agreement among experts and researchers that communication by authorities to 
the public should include explicit information about uncertainties associated with events. It is 
important to ensure that the information provided is consistent and not contradictory, and is 
presented clearly and in an easy to understand manner. However, in contrast, some experts/ 
scientists indicate that providing information about scientific uncertainty can have a negative 
impact on the extent to which the public trusts science, scientists, and scientific institutions; they 
view the general public as unable to conceptualize uncertainties associated with risk management 
processes and so providing the public with information about uncertainty will cause panic and 
confusion regarding the extent and impact of a particular event. Countries covered include 
Canada, China, European Union countries, New Zealand, Thailand, the United States, and 
general globally. Events include bioterrorism, floods, foodborne illness, industrial accident, 
infectious disease, volcanic, and general public health. Preparation, onset, and containment 
phases are covered, with emphasis on preparation, along with evaluation. Low SES minorities, 
pregnant women, children, and people with chronic disease vulnerable populations are included. 


Doyle (2011); Frewer (2003); Holmes 
(2009); Janmaimool (2014); Pappenber- 
ger (2013); Quinn (2008); Taylor-Clark 
(2007); FAO/WHO (2011) FR; Masse 
(2011) FR; Li (2010) CH 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): Low to 
Moderate 

QL (CERQual): Moderate 
MM, CS: Moderate 


To develop messages that contain uncertainty information, it is important to keep in mind the Aldunce (2007); Voorst (2015) 

whole living environment of the intended audience. People’s lives may be full of uncertainties 

due to poverty and not just because of a particular hazard. It should be recognized that issues of 

development and environment are just as central to reduced uncertainty regarding an event as 

messages from authorities. Countries covered include Chile and Indonesia. Events are flood and 

landslide, and the phase is preparation. Low SES vulnerable populations are included. 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): — 
QL (CERQual): — 

MM, CS: Moderate 
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Uncertainty information in messages provided by authorities is generally associated with desira¬ 
ble outcomes but the possibility of some undesirable outcomes needs to be kept in mind. Some 
positive outcomes include: Reduced experienced uncertainty; reduced uncertainty about health 
protection actions; reduced information void, circulation of misinformation and rumors, and 
reliance on sensationalized media stories; improved efficiency, quality, and value of forecasts; and 
improved response to future warnings. However, openly acknowledging uncertainties in messages 
may reduce trust in authorities. But, this decrease is only for a small proportion of the total 
number of message recipients; for the vast majority of message recipients, there is no change in 
their level of trust. Additionally, the uncertainty about specific parameters of an event may 
sometimes leave the public not have enough time to prepare property or move belongings to a 
safe location. Also, the phrasing of the uncertainty information may sometimes be interpreted 
negatively, which may affect response to future risk communication. Countries covered are 
Canada, European Union countries, India, New Zealand, Norway, Russia, South Africa, Thai¬ 
land, the United Kingdom, and the United States. Events are bioterrorism, cyclone/ hurricane, 
earthquake, flood, food contamination, general public health, industrial accident, and infectious 
disease. All four phases along with evaluation covered. Low SES minorities vulnerable popula¬ 
tions included. 


Janmaimool (2014); Johnson (2015); 
McClure (2015); Miles (2003); Morss & 
Hayden (2010); Sharma (2012); Ramos 
(2010); Skinner (2014); Vaughan (2012); 
Duchene (2004) FR; Gryzunova (2012) 
RU; Jakubowski (2004) FR; Lord (2009) 
FR; Pozdnyakov (2011) RU 


QN-CG (GRADE): Low to Moderate 

Low to ModerateQN-DS (GRADE 
Adapted): 

QL (CERQual): Moderate 
MM, CS: Moderate to High 


The public’s understanding of some uncertainty information associated with event likelihood Doyle (2011); Doyle (2014); McClure 

estimates is error prone. This error is true of experts (scientists, non-scientists) as well. The (2015) 

likelihood of event occurrence is rated higher in later intervals than in earlier intervals of a time 

window; the likelihood of event occurrence is not understood as being uniform throughout a time 

window. For example, the likelihood of an event occurrence in a 3-day time window is rated 

higher toward the end interval and lower in the first interval of the window, as opposed to 

uniform across all periods in the time window. Additionally for experts, translation of verbal 

descriptions of event likelihood uncertainty to numerical terms is not fully accurate. Country 

covered is New Zealand. Events are earthquake and volcanic. Phase covered is preparation. No 

vulnerable populations are included. 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): Low to 
Moderate 

QL (CERQual): — 


MM, CS: — 
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Uncertainty of data and knowledge influences interactions within and among groups of experts Downton (2005); Morss (2010); Morss 
and between experts and policy/ decision makers and this affects the decision-making process in (2015); Ramos (2010) 
complex ways. It is important to see a final decision (e.g., evacuation warning) as the end point of 
a chain of decisions that includes a flow of uncertainty information. Experts have to use (scienti¬ 
fically) uncertain data, and in rapidly evolving situations where multiple actors have to make 
interrelated decisions under this uncertainty, there is a greater danger of risk assessment propaga¬ 
ting across individuals in unintended ways. Different people in the decision chain perceive and 
understand uncertainty, and tend to act in face of uncertain information,, differently. For exam¬ 
ple, policy/ decision makers may not fully understand scientific uncertainty and may default to 
their intuitions and experience to make decisions. Additionally, uncertain scientific knowledge is 
entwined with values issues (appropriate standards for public protection), which makes the 
decision chain process even more complex. Countries covered include several European countries 
and the United States. Event is floods, and preparation and onset phases are covered along with 
evaluation. No vulnerable populations are included. 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): - 
QL (CERQual): Moderate 
MM, CS: Moderate to High 


Mass media coverage of an event that emphasizes rapidly changing, contradictory, and conflic- Afifi (2014); Quinn (2008); Rousseau 

ting information, especially that differs from official information from authorities, increases (2008); Taylor-Clark (2007) 

uncertainty in the public, which in turn can lead to several undesirable outcomes. These include: 

lack of trust in authorities and recommended actions; confusion and fear; reduced intentions for 

health protective behaviors such as vaccination; and reduced attention to health risk news. Such 

media coverage also puts a constraint on the ability of frontline health/medical workers to 

address the public’s uncertainty. Countries covered are Canada, France, and the United States. 

Events are bioterrorism, infectious disease, tornado, and general public health. All four phases are 
covered. Low SES minorities vulnerable population included. 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): - 
QL (CERQual): Moderate to High 
MM, CS: Moderate 
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As a result of absent or contradictory and inconsistent information from authorities, medical/ Gesser-Edelsburg (2014); Rousseau 
health care workers and policy makers experience uncertainty and the organizational decision (2008) 
making regarding communication to the public becomes uncertain. Instead of providing transpa¬ 
rent communication regarding the uncertainty surrounding an emergent event, if authorities rush 
to declare a “fact” about the event without adequate information, it can lead to compromised 
decision making and efforts by organizations. Countries covered are Canada, France, and Israel. 

Event is infectious disease. Onset and containment phases are covered. No vulnerable popula¬ 
tions are included. 


QN-CG (GRADE): — 

QN-DS (GRADE Adapted): — 
QL (CERQual): — 

MM, CS: Moderate 


Notes, (back to text p. 12, p. 14) 

Citations-Language: English has no suffix; Arabic (AR); Chinese (CH); French (FR); Russian (RU); Spanish (SP) 

Certainty/ Confidence Evaluation: QN-CG (GRADE) - High; Moderate; Low; Very low; QN-DS (GRADE Adapted) - High; Moderate; Low; Very low; QL (CERQual) - High; 
Moderate; Low; Very low; MM, CS (as appropriate) - High; Moderate; Low; Very low 
1 Only English language studies considered. 
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